Information processing apparatus, information processing method, and information processing program

ABSTRACT

An information processing apparatus including at least one processor, wherein the processor is configured to: acquire a first image obtained by imaging a subject; extract a first region of interest from the first image; and determine whether or not an unsuitable region that is unsuitable for extraction exists, for the extracted first region of interest.

CROSS-REFERENCE TO RELATED APPLICATIONS

The present application claims priority under 35 U.S.C. § 119 toJapanese Patent Application No. 2021-013738, filed on Jan. 29, 2021. Theabove application is hereby expressly incorporated by reference, in itsentirety, into the present application.

BACKGROUND Technical Field

The present disclosure relates to an information processing apparatus,an information processing method, and an information processing program.

Related Art

Conventionally, a doctor has generally made a diagnosis on the basis ofa medical image acquired by an image acquisition apparatus, such as acomputed tomography (CT) apparatus and a magnetic resonance imaging(MRI) apparatus. Further, technology (so-called computer aideddetection/diagnosis (CAD)) in which a computer assists in detecting anddiagnosing a structure, such as an abnormal shadow and a tissue,included in the medical image has also been known. For example,JP2020-032043A describes that a medical image is analyzed by using adiscriminator of which learning has been performed by machine learningand the type of tissue or lesion included in the medical image, that is,the type of observation is specified.

Further, for example, “A Human-Centered Evaluation of a Deep LearningSystem Deployed in Clinics for the Detection of Diabetic Retinopathy”,Emma Beede et al., In CHI, 2020, Paper 589 describes that CAD is notperformed in a case where at least a part of a medical image is unclear,in order to avoid non-detection and misdiagnosis of an abnormal shadow.The unclear medical image may be acquired, for example, in a case whereimaging is not performed in an appropriate environment or a photographerhas a poor imaging technique.

Incidentally, for example, in medical institutions and the like inemerging countries, it may be difficult to acquire a clear medical imagebecause an appropriate environment for capturing a medical image cannotbe created or a photographer has a poor imaging technique. Therefore, inrecent years, there has been a demand for technology capable ofutilizing even an image that is unclear, that is, unsuitable forextraction of a region of interest which may include a structure, suchas an abnormal shadow and a tissue, for diagnosis.

SUMMARY

The present disclosure provides an information processing apparatus, aninformation processing method, and an information processing program bywhich even an image that is unsuitable for extraction of a region ofinterest can be utilized for diagnosis.

According to an aspect of the present disclosure, there is provided aninformation processing apparatus including at least one processor, inwhich the processor acquires a first image obtained by imaging asubject, extracts a first region of interest from the first image, anddetermines whether or not an unsuitable region that is unsuitable forextraction exists, for the extracted first region of interest.

In the above-described aspect, the processor may make the determinationon the basis of a degree of similarity between a shape of the extractedfirst region of interest and a predetermined reference shape for thefirst region of interest.

In the above-described aspect, the processor may make the determination,by using a learned model that is used to determine whether or not theunsuitable region exists in a region of interest which is extracted froman image obtained by imaging a subject, in response to an input of theimage.

In the above-described aspect, the learned model may be a learning modelof which learning has been performed by using, as data for learning, apair of an image obtained by imaging a subject and informationindicating whether or not the unsuitable region exists in a region ofinterest which is extracted from the image.

In the above-described aspect, the processor may specify and present theunsuitable region in the first image in a case where the processordetermines that the unsuitable region exists in the first region ofinterest.

In the above-described aspect, the processor may present a ratio of theunsuitable region to the first region of interest in a case where theprocessor determines that the unsuitable region exists in the firstregion of interest.

In the above-described aspect, the processor may re-extract the firstregion of interest with reduced extraction accuracy in a case where theprocessor determines that the unsuitable region exists in the firstregion of interest.

In the above-described aspect, the processor may detect a structureincluded in the first region of interest, and detect a structureincluded in the unsuitable region with reduced detection accuracy in acase where the processor determines that the unsuitable region exists inthe first region of interest.

In the above-described aspect, the processor may request a second imageincluding a region corresponding to at least a part of the unsuitableregion in a case where the processor determines that the unsuitableregion exists in the first region of interest.

In the above-described aspect, the processor may acquire the secondimage, and combine the first image and the second image to generate athird image.

In the above-described aspect, the processor may combine any one imageof the first image or the second image with a part of the other image sothat the unsuitable region in the one image is complemented by acorresponding region in the other image, to generate the third image.

In the above-described aspect, the processor may select and combine oneimage having better image quality for each plurality of sections in thefirst image and the second image, to generate the third image.

In the above-described aspect, the processor may extract a third regionof interest from the third image, determine whether or not theunsuitable region exists, for the extracted third region of interest,and repeat acquisition of a new image including a region correspondingto at least a part of the unsuitable region and re-combination of thenew image and the third image, until the processor determines that theunsuitable region does not exist in the third region of interest.

In the above-described aspect, the processor may extract a third regionof interest from the third image, and detect a structure included in thethird region of interest.

In the above-described aspect, the processor may acquire the secondimage, extract a second region of interest from the second image, detecta structure included in each of the first region of interest and thesecond region of interest, and combine detection results of thestructures that are detected respectively from the first region ofinterest and the second region of interest.

In the above-described aspect, the processor may determine whether ornot a common unsuitable region that is unsuitable for extraction incommon exists, for the extracted first region of interest and secondregion of interest, and repeat acquisition of a new image including aregion corresponding to at least a part of the common unsuitable region,extraction of a region of interest from the new image, detection of astructure included in the region of interest, and re-combination of adetection result of the structure, until the processor determines thatthe common unsuitable region does not exist.

In the above-described aspect, the first region of interest may be aregion including at least one of the subject, a part of a tissueincluded in the subject, or an abnormal part included in the subject orthe tissue.

In the above-described aspect, the first image may be an image obtainedby at least one of a radiography apparatus, a magnetic resonance imagingapparatus, an ultrasonic apparatus, a fundus photography apparatus, oran endoscope.

According to another aspect of the present disclosure, there is providedan information processing method executed by a computer, the methodincluding: acquiring a first image obtained by imaging a subject;extracting a first region of interest from the first image; anddetermining whether or not an unsuitable region that is unsuitable forextraction exists, for the extracted first region of interest.

According to another aspect of the present disclosure, there is providedan information processing program causing a computer to execute aprocess including: acquiring a first image obtained by imaging asubject; extracting a first region of interest from the first image; anddetermining whether or not an unsuitable region that is unsuitable forextraction exists, for the extracted first region of interest.

With the information processing apparatus, the information processingmethod, and the information processing program of the present disclosureaccording to the above-described aspects, even an image that isunsuitable for extraction of a region of interest can be utilized fordiagnosis.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic configuration diagram of an information processingsystem.

FIG. 2 is a block diagram showing an example of a hardware configurationof an information processing apparatus.

FIG. 3 is a block diagram showing an example of a functionalconfiguration of an information processing apparatus according to afirst embodiment.

FIG. 4 is an example of a clear medical image.

FIG. 5 is an example of an unclear first image.

FIG. 6 is an example of a screen that is presented.

FIG. 7 is a flowchart showing an example of determination processing.

FIG. 8 is a block diagram showing an example of a functionalconfiguration of an information processing apparatus according to secondand third embodiments.

FIG. 9 is an example of another screen that is presented.

FIG. 10 is an example of an unclear second image.

FIG. 11 is an example of a combined third image.

FIG. 12 is an example of another screen that is presented.

FIG. 13 is a flowchart showing an example of image combinationprocessing.

FIG. 14 is an example of another screen that is presented.

FIG. 15 is a flowchart showing an example of detection resultcombination processing.

FIG. 16 is a diagram showing an example of a pseudo image that is inputto a learning model.

FIG. 17 is a flowchart showing an example of learning processing.

DETAILED DESCRIPTION

Hereinafter, exemplary embodiments of technology according to thepresent disclosure will be described in detail with reference to thedrawings.

First Embodiment

An example of a configuration of an information processing system 1according to the present embodiment will be described with reference toFIG. 1. As shown in FIG. 1, the information processing system 1 includesan information processing apparatus 10 and an image acquisitionapparatus 2. The information processing apparatus 10 and the imageacquisition apparatus 2 can communicate with each other by wired orwireless communication.

The image acquisition apparatus 2 is an apparatus (so-called modality)that acquires an image obtained by imaging a subject. In the presentembodiment, description will be made by using a medical image as aspecific example of the image that is acquired by the image acquisitionapparatus 2. As the image acquisition apparatus 2, at least one of aradiography apparatus, a magnetic resonance imaging apparatus, anultrasonic apparatus, a fundus photography apparatus, or an endoscopecan be applied, and an appropriate combination thereof may be applied.

Meanwhile, for example, in order to clearly image the fundus of an eye,it is necessary to perform imaging in a dark place, but medicalinstitutions in emerging countries cannot create a sufficient darkplace. In this case, an unclear medical image may be captured. Inaddition, for example, in order to clearly image a breast inmammography, it is necessary to sufficiently compress the breast, butthe breast cannot be sufficiently compressed depending on thepositioning technique of a photographer and the shape of the breast. Inthis case, an unclear medical image may be captured. In recent years,there has been a demand for technology in which even such an unclearimage can be utilized for diagnosis.

Therefore, the information processing apparatus 10 according to thepresent embodiment has a function of determining whether or not anunclear region is included in the medical image. The term “unclear”means a case where a pixel value such as hue, saturation, brightness,and lightness represented by each pixel of the medical image does notsatisfy a predetermined reference value. The unclear region may begenerated, for example, in a case where the pixel becomes darker thanthe reference value due to insufficient light amount, or the pixelbecomes brighter than the reference value due to ambient light.Hereinafter, an example of the configuration of the informationprocessing apparatus 10 according to the present embodiment will bedescribed.

First, an example of a hardware configuration of the informationprocessing apparatus 10 according to the present embodiment will bedescribed with reference to FIG. 2. As shown in FIG. 2, the informationprocessing apparatus 10 includes a central processing unit (CPU) 21, anon-volatile storage unit 22, and a memory 23 as a temporary storagearea. In addition, the information processing apparatus 10 includes adisplay 24, such as a liquid crystal display, an input unit 25, such asa keyboard and a mouse, and a network I/F 26 that performs wired orwireless communication with the image acquisition apparatus 2 and anexternal network (not shown). The CPU 21, the storage unit 22, thememory 23, the display 24, the input unit 25, and the network I/F 26 areconnected to one another via a bus 28 such as a system bus and a controlbus so that various information can be exchanged.

The storage unit 22 is implemented with, for example, a storage mediumsuch as a hard disk drive (HDD), a solid state drive (SSD), and a flashmemory. The information processing program 27 according to the presentembodiment is stored in the storage unit 22. The CPU 21 reads out theinformation processing program 27 from the storage unit 22, extracts theprogram to the memory 23, and executes the extracted informationprocessing program 27. The CPU 21 is an example of a processor of thepresent disclosure. As the information processing apparatus 10, forexample, various computers such as a console of the image acquisitionapparatus 2, a workstation, a server computer, and a personal computercan be applied.

Next, an example of a functional configuration of the informationprocessing apparatus 10 according to the present embodiment will bedescribed with reference to FIG. 3. As shown in FIG. 3, the informationprocessing apparatus 10 includes an acquisition unit 11, an extractionunit 12, a determination unit 13, a detection unit 14, and apresentation unit 15. The CPU 21 executes the information processingprogram 27 to function as the acquisition unit 11, the extraction unit12, the determination unit 13, the detection unit 14, and thepresentation unit 15.

The acquisition unit 11 acquires a medical image obtained by imaging asubject, from the image acquisition apparatus 2. As a target structurefor which detection and diagnosis are desired, a region of interestincluding at least one of the subject, a part of a tissue included inthe subject, or an abnormal part included in the subject or the tissueis included in the medical image of the present embodiment. Examples ofthe subject include a human body and various organs of the human bodysuch as the fundus of an eye, lungs, breast, stomach, liver, heart, andbrain. Examples of the tissue include elements that constitute variousorgans such as blood vessels, nerves, and muscles. Examples of theabnormal part include a lesion such as tumors, injuries, defects,nodules, and inflammation, and an abnormality.

FIG. 4 shows a medical image G0 as an example of a clear medical image.The medical image G0 is a clear image of the fundus of the eye obtainedby the fundus photography apparatus. In the image of the fundus of theeye, since the abnormal part such as the lesion may be included in theentire imaging range including the fovea centralis, macula, optic nervehead, and blood vessels, the entire imaging range corresponds to aregion of interest A0. In the example of FIG. 4, as an example of theabnormal part, abnormal shadows S1 and S2 are included in the region ofinterest A0.

On the other hand, FIG. 5 shows a first image G1 as an example of anunclear medical image. The first image G1 is an image of the same fundusof the eye as that in the medical image G0, but a part of the imagingrange is unclear and an unsuitable region N1 that is unsuitable forextraction of an original first region of interest A1 (that is, theentire imaging range) exists. The unsuitable region N1 originallyincludes the abnormal shadow S1, but the unclearness of the unsuitableregion N1 makes the detection thereof difficult. Hereinafter,description will be made assuming that the acquisition unit 11 acquiresthe first image G1.

The extraction unit 12 extracts the first region of interest A1 from thefirst image G1 acquired by the acquisition unit 11. As a method ofextracting the first region of interest A1, a method using known imageprocessing, a method using artificial intelligence (AI) technology, orthe like can be appropriately applied. For example, the first image G1may be binarized, the background may be removed, the edges of eachstructure may be emphasized, and the outline of the imaging range may bespecified, so that the first region of interest A1 may be extracted.Alternatively, the extraction unit 12 may extract the imaging range(that is, the first region of interest A1) from the first image G1, forexample, by using a learned model that has learned to extract and outputthe imaging range in response to an input of the image of the fundus ofthe eye.

The detection unit 14 detects an abnormal shadow included in the firstregion of interest A1 extracted by the extraction unit 12. In theexample of the first image G1 of FIG. 5, the abnormal shadow S2 isdetected by the detection unit 14, but the abnormal shadow S1 is notdetected because the abnormal shadow S1 is included in the unclearunsuitable region N1. As a method of detecting the abnormal shadow, aknown CAD technology can be appropriately applied.

The determination unit 13 determines whether or not the unsuitableregion N1 that is unsuitable for extraction exists, for the first regionof interest A1 extracted by the extraction unit 12. Various methods canbe applied as the determination method. For example, the determinationunit 13 may make the determination on the basis of a degree ofsimilarity between the shape of the first region of interest A1extracted by the extraction unit 12 and a predetermined reference shapefor the first region of interest A1. Since the region of interest is theentire imaging range in a case of an image of the fundus of the eye, thereference shape can be predetermined to be substantially circular.Therefore, in a case where the degree of similarity between the shape ofthe outline of the first region of interest A1 extracted by theextraction unit 12 and the substantially circular reference shape is apredetermined threshold value or less (that is, the deviation is large),the unsuitable region N1 may be determined to exist. As a determinationmethod based on the degree of similarity, known matching techniques suchas matching by a feature amount and template matching can beappropriately applied.

Alternatively, for example, AI technology is applied to thedetermination, and determination may be made, by using a learned modelthat is used to determine whether or not the unsuitable region exists inthe region of interest extracted from the medical image, in response toan input of the medical image. The learned model in this case may be amodel of which learning has been performed by unsupervised learning or amodel that has learned to, for example, cluster medical images accordingto the existence or nonexistence of unsuitable regions. Alternatively,the learned model may be a model of which learning has been performed bysupervised learning or a model of which learning has been performed byusing, for example, a pair of the medical image and informationindicating whether or not the unsuitable region exists in the region ofinterest extracted from the medical image, as data for learning.

FIG. 6 shows an example of a screen D1 that is presented on the display24 by the presentation unit 15. As shown in FIG. 6, the presentationunit 15 presents the abnormal shadow S2 detected by the detection unit14 with annotation M added onto the first image G1.

Further, in a case where the determination unit 13 determines that theunsuitable region N1 exists in the first region of interest A1, thepresentation unit 15 specifies and presents the unsuitable region N1 inthe first image G1. In the example of FIG. 6, the presentation unit 15presents the unsuitable region N1 surrounded by a thick line in thefirst image G1. The method of presenting the unsuitable region N1 is notlimited thereto, and the unsuitable region N1 need only be emphasized soas to be discriminable, for example, by using a different line type (forexample, the line thickness, color, solid line, and dotted line) fromthe other region, or annotation added. On the contrary, a region exceptfor the unsuitable region N1 in the first region of interest A1 may beemphasized, to make the unsuitable region N1 discriminable.

Further, in a case where the determination unit 13 determines that theunsuitable region N1 exists in the first region of interest A1, thepresentation unit 15 may present a ratio of the unsuitable region N1 tothe first region of interest A1. In the example of FIG. 6, theunsuitable region N1 is illustrated as an “inexecutable” region in whichabnormal shadow detection processing by the detection unit 14 cannot beperformed. Further, the region except for the unsuitable region N1 inthe first region of interest A1 is illustrated as an“execution-completed” region in which the detection of the abnormalshadow by the detection unit 14 is completed.

Next, an operation of the information processing apparatus 10 accordingto the present embodiment will be described with reference to FIG. 7.The CPU 21 executes the information processing program 27, so thatdetermination processing shown in FIG. 7 is executed. The determinationprocessing shown in FIG. 7 is executed, for example, in a case where auser gives an instruction on the start of the processing via the inputunit 25.

In Step S10, the acquisition unit 11 acquires the first image G1 fromthe image acquisition apparatus 2. In Step S11, the extraction unit 12extracts the first region of interest A1 from the first image G1acquired in Step S10. In Step S12, the detection unit 14 detects theabnormal shadow included in the first region of interest A1 extracted inStep S11. In Step S13, the determination unit 13 determines whether ornot the unsuitable region N1 that is unsuitable for extraction exists,for the first region of interest A1 extracted in Step S11.

In a case where the unsuitable region N1 exists (that is, in a casewhere affirmative determination is made in Step S13), the processproceeds to Step S14, and the presentation unit 15 specifies andpresents the unsuitable region N1 in the first image G1 together withthe abnormal shadow detected in Step S12. On the other hand, in a casewhere the unsuitable region N1 does not exist (that is, in a case wherenegative determination is made in Step S13), the process proceeds toStep S15, and the presentation unit 15 presents only the abnormal shadowdetected in Step S12. When Step S14 or S15 is completed, thedetermination processing ends. After Step S14, image combinationprocessing according to a second embodiment and/or detection resultcombination processing according to a third embodiment, which will bedescribed later, may be performed.

As described above, the information processing apparatus 10 according tothe first embodiment comprises at least one processor, and the processoracquires the first image G1 obtained by imaging the subject, extractsthe first region of interest A1 from the first image G1, and determineswhether or not the unsuitable region N1 that is unsuitable forextraction exists, for the extracted first region of interest A1. Thatis, the information processing apparatus 10 determines the existence ornonexistence of an unclear region in the first region of interest A1that may include a target structure for which detection and diagnosis ofabnormal shadows and the like are desired. Therefore, even the unclearfirst image G1 can be utilized for diagnosis on the basis of therecognition of the existence of an unclear region in the first image G1.

In the first embodiment, the detection unit 14 may detect only theabnormal shadow included in the region except for the unsuitable regionN1 in the first region of interest A1, and may not detect the abnormalshadow in the unsuitable region N1. This is because the unclearunsuitable region N1 has a higher possibility of non-detection anderroneous detection of abnormal shadows as compared with other regions,and the reliability of the detection result is low.

On the other hand, in the first embodiment, in a case where thedetermination unit 13 determines that the unsuitable region N1 exists inthe first region of interest A1, the detection unit 14 may detect thestructure such as abnormal shadows included in the unsuitable region N1with reduced detection accuracy. The phrase “detection with reduceddetection accuracy” means that the possibility of non-detection anderroneous detection of abnormal shadows is allowed, and abnormal shadowsare detected even in a case where the reliability is low. In this case,it is preferable that the presentation unit 15 presents that thedetection accuracy of the abnormal shadows is reduced for the unsuitableregion N1. Further, it is preferable to give presentation as describedabove particularly in a case where the ratio of the unsuitable region N1to the first region of interest A1 is a predetermined threshold value ormore (for example, 20% or more). This is because if detection of theabnormal shadow is not performed in the unsuitable region N1 in a casewhere the ratio of the unsuitable region N1 is high, the user isrequired to visually confirm the abnormal shadow for many parts of thefirst image G1 and the advantages of CAD are lost. According to such anaspect, the detection result of the abnormal shadow for the entire firstimage G1 can be utilized for diagnosis on the basis of the recognitionof the reduced detection accuracy for the unclear region.

Further, in the first embodiment, an aspect in which the detection unit14 detects the abnormal shadow has been described, but the informationprocessing apparatus 10 according to the present embodiment may not havethe function of the detection unit 14 (that is, the CAD function), andthe user may visually confirm the abnormal shadow. According to such anaspect, in a case where the user is made to recognize the existence ofthe unclear region in the first image G1, oversight of the abnormalshadow by the user can be suppressed. Therefore, even the unclear firstimage G1 can be utilized for diagnosis by the user.

Further, in the first embodiment, in a case where the determination unit13 determines that the unsuitable region N1 exists in the first regionof interest A1, the extraction unit 12 may re-extract the first regionof interest A1 with reduced extraction accuracy. The phrase“re-extraction with reduced extraction accuracy” means that it isallowed that other regions (for example, the background portion in FIG.5) may be extracted as the first region of interest A1, and conditionsin a case where the first region of interest A1 is extracted are changedso that the unsuitable region N1 is reduced. The conditions aredetermined by, for example, the brightness value of pixels.

Further, in this case, it is preferable that the presentation unit 15presents that the extraction accuracy of the first region of interest A1is reduced. Further, it is preferable to give presentation as describedabove particularly in a case where the ratio of the unsuitable region N1to the first region of interest A1 is a predetermined threshold value ormore (for example, 20% or more). This is because if many parts of thefirst image G1 are determined as the unsuitable region N1, it isdifficult to complement the unsuitable region N1 (details will bedescribed later). According to such an aspect, in a case where the useris made to recognize the reduced extraction accuracy of the first regionof interest A1, oversight of the abnormal shadow by the user can besuppressed. Therefore, even the unclear first image G1 can be utilizedfor diagnosis by the user.

Second Embodiment

In the first embodiment, determination is made whether or not theunsuitable region N1 exists in the first image G1. In a case wheredetermination is made that the unsuitable region N1 exists in the firstimage G1, the information processing apparatus 10 according to thepresent embodiment has a function of making a medical image that isdifferent from the first image G1 complement the unsuitable region N1.Hereinafter, an example of the configuration of the informationprocessing apparatus 10 according to the present embodiment will bedescribed, but duplicate description will be omitted for the sameconfiguration and operation as those of the first embodiment.

An example of a functional configuration of the information processingapparatus 10 according to the present embodiment will be described withreference to FIG. 8. As shown in FIG. 8, the information processingapparatus 10 according to the present embodiment includes a combinationunit 16, in addition to the same acquisition unit 11, extraction unit12, determination unit 13, detection unit 14, and presentation unit 15as in the first embodiment. The CPU 21 executes the informationprocessing program 27 to function as the acquisition unit 11, theextraction unit 12, the determination unit 13, the detection unit 14,the presentation unit 15, and the combination unit 16.

FIG. 9 shows an example of a screen D2 that is presented on the display24 by the presentation unit 15. As shown in FIG. 9, in a case where thedetermination unit 13 determines that the unsuitable region N1 exists inthe first region of interest A1 of the first image G1, the presentationunit 15 requests a second image G2 including a region corresponding toat least a part of the unsuitable region N1.

The acquisition unit 11 acquires the second image G2 from the imageacquisition apparatus 2. FIG. 10 shows the second image G2. The secondimage G2 is an image of the same fundus of the eye as that in themedical image G0, and a region N12 (shown by the broken line)corresponding to the unsuitable region N1 of the first image G1 isclearly reflected. On the other hand, a part of the imaging range isunclear, and an unsuitable region N2 that is unsuitable for theextraction of an original second region of interest A2 (that is, theentire imaging range) also exists. The unsuitable region N2 originallyincludes an abnormal shadow S2, but the unclearness of the unsuitableregion N2 makes the detection thereof difficult.

The combination unit 16 combines the first image G1 and the second imageG2 acquired by the acquisition unit 11, to generate a third image G3.Specifically, the combination unit 16 combines any one image of thefirst image G1 or the second image G2 with a part of the other image sothat the unsuitable region in the one image is complemented by acorresponding region in the other image, to generate the third image G3.In the third image G3 shown in FIG. 11, a part of the second image G2 iscombined into the first image G1 so that the unsuitable region N1 in thefirst image G1 is complemented by the region N12 corresponding to theunsuitable region N1 in the second image G2.

The extraction unit 12 extracts a third region of interest A3 from thethird image G3 combined by the combination unit 16, by using the samemethod as a method in which the first region of interest A1 is extractedfrom the first image G1.

The determination unit 13 determines whether or not an unsuitable regionexists, for the third region of interest A3 extracted by the extractionunit 12. For example, in a case where a part of the region N12 of thesecond image G2 corresponding to the unsuitable region N1 of the firstimage G1 is unclear, the second image G2 alone cannot complement thefirst image G1. Therefore, the CPU 21 repeats request and acquisition ofa new image including a region corresponding to at least a part of theunsuitable region and re-combination of the new image and the thirdimage G3, until the determination unit 13 determines that the unsuitableregion does not exist in the third region of interest A3. The request,acquisition, and combination of the new image are performed in the samemanner as the request, acquisition, and combination of the second imageG2 described above.

In a case where the determination unit 13 determines that the unsuitableregion does not exist in the third region of interest A3, the detectionunit 14 detects the abnormal shadow included in the third region ofinterest A3 extracted by the extraction unit 12, by using the samemethod as a method in which the abnormal shadow included in the firstregion of interest A1 is detected. In the example of the third image G3of FIG. 11, the detection unit 14 can detect both the abnormal shadowsS1 and S2.

FIG. 12 shows an example of a screen D3 that is presented on the display24 by the presentation unit 15. As shown in FIG. 12, the presentationunit 15 presents the abnormal shadows S1 and S2 detected by thedetection unit 14 with annotation M added onto the third image G3.

Next, an operation of the information processing apparatus 10 accordingto the present embodiment will be described with reference to FIG. 13.The CPU 21 executes the information processing program 27, so that imagecombination processing shown in FIG. 13 is executed. The imagecombination processing shown in FIG. 13 is executed after Step S14 inthe flowchart of FIG. 7. That is, in a case where determination is madein the determination processing of the first embodiment that theunsuitable region N1 exists in the first image G1, the processing isexecuted.

In Step S31, the presentation unit 15 requests the second image G2including a region corresponding to at least a part of the unsuitableregion N1 specified in Step S14. In Step S32, the acquisition unit 11acquires the second image G2 from the image acquisition apparatus 2. InStep S33, the combination unit 16 combines the first image G1 acquiredin Step S10 and the second image G2 acquired in Step S32 to generate thethird image G3.

In Step S34, the extraction unit 12 extracts the third region ofinterest A3 from the third image G3 combined in Step S33. In Step S35,the determination unit 13 determines whether or not the unsuitableregion that is unsuitable for extraction exists, for the third region ofinterest A3 extracted in Step S34.

In a case where the unsuitable region exists (that is, in a case whereaffirmative determination is made in Step S35), the process proceeds toStep S36, and the presentation unit 15 requests a new image including aregion corresponding to at least a part of the unsuitable region. InStep S37, the acquisition unit 11 acquires the new image from the imageacquisition apparatus 2. In Step S38, the combination unit 16re-combines the new image acquired in Step S37 and the third image G3combined in Step S33. When Step S38 is completed, the process returns toStep S34. That is, the processing of Steps S34 to S38 is repeated untildetermination is made in Step S35 that the unsuitable region does notexist in the third region of interest A3.

On the other hand, in a case where the unsuitable region does not exist(that is, in a case where negative determination is made in Step S35),the process proceeds to Step S39, and the detection unit 14 detects theabnormal shadow included in the third region of interest A3 extracted inStep S34. In Step S40, the presentation unit 15 presents the abnormalshadow detected in Step S39, and the image combination processing ends.

As described above, the information processing apparatus 10 according tothe second embodiment comprises at least one processor, and in a casewhere determination is made that the unsuitable region N1 exists in thefirst region of interest A1 of the first image G1, the processorrequests the second image including a region corresponding to at least apart of the unsuitable region N1 and combines the first image G1 and thesecond image G2. That is, in a case where determination is made that theunsuitable region N1 exists in the first image G1, the informationprocessing apparatus 10 makes the second image G2 that is different fromthe first image G1 complement the unsuitable region N1. Therefore, eventhe first image G1 and the second image G2 each of which is unclear canbe utilized for diagnosis.

In the second embodiment, an aspect in which the unsuitable region inany one image of the first image G1 or the second image G2 iscomplemented by the other image by the combination unit 16 has beendescribed, but the present disclosure is not limited thereto. Forexample, the combination unit 16 may select and combine one image havingbetter image quality for each plurality of sections in the first imageG1 and the second image G2, to generate the third image G3. The term“section” means, for example, a pixel and a block that is constituted ofplural pixels. The quality of the image quality can be evaluated on thebasis of, for example, a pixel value such as hue, saturation,brightness, and lightness represented by each pixel. Further, with thecombination of the aspects, the combination unit 16 may select andcombine one image having better image quality for each section, for aregion except for the unsuitable region while making the other imagecomplement the unsuitable region.

Further, in the second embodiment, for example, particularly in a casewhere the combination unit 16 repeats the re-combination based on thenew image more than a predetermined number of times (for example, 3times), the extraction unit 12 may re-extract the third region ofinterest A3 with reduced extraction accuracy. This is for finishing theprocessing in a case where the unsuitable region is not complementedeven if the re-combination is repeated. In this case, it is preferablethat the presentation unit 15 presents that the extraction accuracy ofthe third region of interest A3 is reduced.

Further, in the second embodiment, an aspect in which the acquisitionand combination of the new image are repeated until the unsuitableregion does not exist in the third image G3 has been described, but thepresent disclosure is not limited thereto. In a case where thecombination unit 16 performs combination at least once, the combinationprocessing may be finished even if the unsuitable region exists. Forexample, in a case where a limit based on the number of times ofcombination (for example, 3 times) may be set and the number of times ofcombination exceeds the limit, the processing may be finished even ifthe unsuitable region exists. Further, for example, in a case where theratio of the unsuitable region to the third region of interest A3 of thethird image G3 is a predetermined threshold value or less (for example,5% or less), the processing may be finished even if the unsuitableregion exists. In these cases, the presentation unit 15 may present theratio of the unsuitable region to the third region of interest A3.

Further, in a case where the processing is finished in a state in whichthe unsuitable region exists, the structure such as an abnormal shadowincluded in the third region of interest A3 may be detected with reduceddetection accuracy. In this case, it is preferable that the presentationunit 15 presents that the detection accuracy of the abnormal shadows isreduced for the unsuitable region. Further, it is preferable to givepresentation as described above particularly in a case where the ratioof the unsuitable region to the third region of interest A3 is apredetermined threshold value or more (for example, 20% or more). Thisis because if detection of the abnormal shadow is not performed in theunsuitable region in a case where the ratio of the unsuitable region ishigh, the user is required to visually confirm the abnormal shadow formany parts of the third image G3 and the advantages of CAD are lost.According to such an aspect, the detection result of the abnormal shadowfor the entire third image G3 can be utilized for diagnosis on the basisof the recognition of the reduced detection accuracy for the unclearregion.

Further, in the second embodiment, an aspect in which the detection unit14 detects the abnormal shadow has been described, but the informationprocessing apparatus 10 according to the present embodiment may not havethe function of the detection unit 14 (that is, the CAD function), andthe user may visually confirm the abnormal shadow. In such an aspect,even the first image G1 and the second image G2 each of which is unclearcan also be utilized for diagnosis by the user.

Further, in the second embodiment, an aspect in which the imagecombination processing is performed in a case where determination ismade in the determination processing of the first embodiment that theunsuitable region N1 exists in the first image G1 has been described,but the present disclosure is not limited thereto. For example, in thefirst image G1, in a case where the ratio of the unsuitable region N1 tothe first region of interest A1 is a predetermined threshold value ormore (for example, 20% or more), the image combination processingaccording to the present embodiment may be performed.

Third Embodiment

In the second embodiment, the abnormal shadow is detected on the basisof the third image G3 in which the first image G1 and the second imageG2 are combined. The information processing apparatus 10 according tothe present embodiment has a function of detecting the abnormal shadowon the basis of each of the first image G1 and the second image G2 andcombining the detection results. Hereinafter, an example of theconfiguration of the information processing apparatus 10 according tothe present embodiment will be described, but duplicate description willbe omitted for the same configuration and operation as those of thefirst and second embodiments.

As in the second embodiment, an example of the functional configurationof the information processing apparatus 10 according to the presentembodiment will be described with reference to FIG. 8. As shown in FIG.8, the information processing apparatus 10 according to the presentembodiment includes the combination unit 16, in addition to the sameacquisition unit 11, extraction unit 12, determination unit 13,detection unit 14, and presentation unit 15 as in the first embodiment.The CPU 21 executes the information processing program 27 to function asthe acquisition unit 11, the extraction unit 12, the determination unit13, the detection unit 14, the presentation unit 15, and the combinationunit 16.

FIG. 9 shows an example of the screen D2 that is presented on thedisplay 24 by the presentation unit 15. As shown in FIG. 9, in a casewhere the determination unit 13 determines that the unsuitable region N1exists in the first region of interest A1 of the first image G1, thepresentation unit 15 requests the second image G2 including a regioncorresponding to at least a part of the unsuitable region N1. Theacquisition unit 11 acquires the second image G2 (see FIG. 10) from theimage acquisition apparatus 2. Since the description of the second imageG2 is the same as that of the second embodiment, the description thereofwill be omitted.

The extraction unit 12 extracts the second region of interest A2 fromthe second image G2 acquired by the acquisition unit 11, by using thesame method as a method in which the first region of interest A1 isextracted from the first image G1. The detection unit 14 detects theabnormal shadow included in each of the first region of interest A1 andthe second region of interest A2 extracted by the extraction unit 12, byusing the same method as a method in which the abnormal shadow includedin the first region of interest A1 is detected. In the examples of FIGS.5 and 10, the detection unit 14 detects the abnormal shadow S2 from thefirst image G1 and detects the abnormal shadow S1 from the second imageG2.

The combination unit 16 combines the detection results of the abnormalshadows that are detected respectively from the first region of interestA1 and the second region of interest A2, by the detection unit 14. Thatis, the combination unit 16 combines the detection results of abnormalshadows obtained from plural different images as the detection resultsof abnormal shadows for the same subject.

The determination unit 13 determines whether or not a common unsuitableregion that is unsuitable for extraction in common exists, for the firstregion of interest A1 and the second region of interest A2 extracted bythe extraction unit 12. The existence of the common unsuitable regionmay lead to oversight of the abnormal shadow. Therefore, the CPU 21repeats request and acquisition of a new image including a regioncorresponding to at least a part of the common unsuitable region,extraction of a region of interest from the new image, detection of theabnormal shadow included in the region of interest, and re-combinationof the detection result of the abnormal shadow and the detection resultof the abnormal shadow detected so far, until the determination unit 13determines that the common unsuitable region does not exist. The requestand acquisition of the new image, the extraction of the region ofinterest, and the detection of the abnormal shadow are performed in thesame manner as the request and acquisition of the second image G2, theextraction of the region of interest, and the detection of the abnormalshadow described above.

FIG. 14 shows an example of a screen D4 that is presented on the display24 by the presentation unit 15. As shown in FIG. 14, the presentationunit 15 presents the detection results of the abnormal shadows combinedby the combination unit 16 with annotation M added onto each image. Asin the second embodiment, the combination unit 16 may generate an imagein which the first image G1 and the second image G2 are combined, andthe presentation unit 15 may present the detection results of theabnormal shadows combined by the combination unit 16 on the image (thatis, one image) with annotation M added thereto.

Next, an operation of the information processing apparatus 10 accordingto the present embodiment will be described with reference to FIG. 15.The CPU 21 executes the information processing program 27, so thatdetection result combination processing shown in FIG. 15 is executed.The detection result combination processing shown in FIG. 15 is executedafter Step S14 in the flowchart of FIG. 7. That is, in a case wheredetermination is made in the determination processing of the firstembodiment that the unsuitable region N1 exists in the first image G1,the processing is executed.

In Step S51, the presentation unit 15 requests the second image G2including a region corresponding to at least a part of the unsuitableregion N1 specified in Step S14. In Step S52, the acquisition unit 11acquires the second image G2 from the image acquisition apparatus 2. InStep S53, the extraction unit 12 extracts the second region of interestA2 from the second image G2 acquired in Step S52. In Step S54, thedetection unit 14 detects the abnormal shadow included in the secondregion of interest A2 extracted in Step S53.

In Step S55, the combination unit 16 combines the detection result ofthe abnormal shadow included in the first region of interest A1 detectedin Step S12 and the detection result of the abnormal shadow included inthe second region of interest A2 detected in Step S54. In Step S56, thedetermination unit 13 determines whether or not a common unsuitableregion that is unsuitable for extraction in common exists, for the firstregion of interest A1 extracted in Step S11 and the second region ofinterest A2 extracted in Step S53.

In a case where the common unsuitable region exists (that is, in a casewhere affirmative determination is made in Step S56), the processing ofSteps S51 to S56 is performed, for the new image including a regioncorresponding to at least a part of the common unsuitable region. Thatis, the processing of Steps S51 to S56 is repeated until determinationis made in Step S56 that the common unsuitable region does not exist.

On the other hand, in a case where the common unsuitable region does notexist (that is, in a case where negative determination is made in StepS56), the process proceeds to Step S57, and the presentation unit 15presents the detection results of the abnormal shadows combined in StepS55. When Step S57 is completed, the detection result combinationprocessing ends.

As described above, the information processing apparatus 10 according tothe third embodiment comprises at least one processor, and in a casewhere determination is made that the unsuitable region N1 exists in thefirst region of interest A1 of the first image G1, the processorrequests the second image including a region corresponding to at least apart of the unsuitable region N1. Further, the information processingapparatus 10 extracts the second region of interest A2 from the secondimage G2, detects the abnormal shadow included in each of the firstregion of interest A1 and the second region of interest A2, and combinesthe detection results of the abnormal shadows. That is, in a case wheredetermination is made that the unsuitable region N1 exists in the firstimage G1, the information processing apparatus 10 makes the second imageG2 that is different from the first image G1 complement the detectionresult of the abnormal shadow. Therefore, even the first image G1 andthe second image G2 each of which is unclear can be utilized fordiagnosis.

In the third embodiment, for example, particularly in a case where thecombination unit 16 repeats the re-combination of the detection resultof the abnormal shadow included in the new image more than apredetermined number of times (for example, 3 times), the extractionunit 12 may extract a region of interest of the new image with reducedextraction accuracy. This is for finishing the processing in a casewhere the common unsuitable region does not disappear even if there-combination is repeated. In this case, it is preferable that thepresentation unit 15 presents that the extraction accuracy of the regionof interest is reduced.

Further, in the third embodiment, an aspect in which the combination ofthe detection result of the abnormal shadow based on the new image isrepeated until the common unsuitable region does not exist has beendescribed, but the present disclosure is not limited thereto. In a casewhere the combination unit 16 performs combination of the detectionresult at least once, the combination processing may be finished even ifthe common unsuitable region exists. For example, in a case where alimit based on the number of times of combination (for example, 3 times)is set and the number of times of combination exceeds the limit, theprocessing may be finished even if the common unsuitable region exists.Further, for example, in a case where the ratio of the common unsuitableregion to the first region of interest A1 or the second region ofinterest A2 is a predetermined threshold value or less (for example, 5%or less), the processing may be finished even if the common unsuitableregion exists.

Further, in a case where the processing is finished in a state in whichthe common unsuitable region exists, the structure such as an abnormalshadow included in the common unsuitable region may be detected withreduced detection accuracy. In this case, it is preferable that thepresentation unit 15 presents that the detection accuracy of theabnormal shadows is reduced for the common unsuitable region. Further,it is preferable to give presentation as described above particularly ina case where the ratio of the common unsuitable region to the firstregion of interest A1 or the second region of interest A2 is apredetermined threshold value or more (for example, 20% or more). Thisis because if detection of the abnormal shadow is not performed in thecommon unsuitable region in a case where the ratio of the commonunsuitable region is high, the user is required to visually confirm theabnormal shadow for many parts of the first image G1 and the secondimage G2 and the advantages of CAD are lost. According to such anaspect, the detection result of the abnormal shadow for the entire firstimage G1 and second image G2 can be utilized for diagnosis on the basisof the recognition of the reduced detection accuracy for the unclearregion.

Further, in the third embodiment, an aspect in which the detectionresult combination processing is performed in a case where determinationis made in the determination processing of the first embodiment that theunsuitable region N1 exists in the first image G1 has been described,but the present disclosure is not limited thereto. For example, in thefirst image G1, in a case where the ratio of the unsuitable region N1 tothe first region of interest A1 is a predetermined threshold value ormore (for example, 20% or more), the detection result combinationprocessing according to the present embodiment may be performed.

Fourth Embodiment

In the first to third embodiments, an aspect in which the extractionunit 12 extracts the region of interest from the medical image has beendescribed. As described above, as a method of extracting the region ofinterest by the extraction unit 12, a learned model that has learned toextract and output the region of interest in response to the input ofthe medical image may be used. In this case, the learned model isrequired to be able to accurately extract the region of interest even ina case where an unclear medical image as shown in FIGS. 5 and 10 isinput.

As one method for improving the accuracy of the learned model, there isa method of performing learning by using unclear medical images havingvarious patterns, as data for learning. However, it has been difficultfor the image acquisition apparatus 2 to acquire a sufficient number andpatterns of unclear medical images. Therefore, in the presentembodiment, an object thereof is to improve the accuracy of a learningmodel by using an unclear medical image that is intentionally generatedas data for learning.

As an example, an aspect in which the information processing apparatus10 according to the present embodiment causes a learning model 4 that isused by the extraction unit 12 to perform learning by unsupervisedlearning will be described. The learning model 4 is a model thatincludes a deep learning model such as a convolutional neural network(CNN), a fully convolutional network (FCN), and U-Net, and that haslearned to extract and output a region of interest in response to theinput of a medical image. Further, as such a model, for example, thetechnology described in JP2020-032043A, JP2019-088458A, andJP2020-114302A may be applied.

The CPU 21 acquires an original image obtained by imaging the subject,from the image acquisition apparatus 2. That is, the original image isan image obtained by at least one image acquisition apparatus 2 of aradiography apparatus, a magnetic resonance imaging apparatus, anultrasonic apparatus, a fundus photography apparatus, or an endoscope.Further, the original image includes a region of interest including atleast one structure, such as the subject, a part of a tissue included inthe subject, and an abnormal part included in the subject or the tissue.Hereinafter, as an example of the original image, an example using theclear medical image G0 shown in FIG. 4 will be described. Since themedical image G0 is as described above, the description thereof will beomitted.

The CPU 21 changes the pixel value of at least a part of the medicalimage G0 to generate a pseudo image. The pixel value is a valueindicating at least one of hue, saturation, brightness, or lightnessrepresented by each pixel in the medical image G0. For example, in acase where the lightness and contrast of the medical image G0 arechanged or blurring and noise are given thereto, the pixel value of eachpixel is changed. Note that, the CPU 21 does not change the resolutionin generating the pseudo image.

A specific example of the pseudo image will be described with referenceto FIG. 16. Plural the pseudo images P1 to P5 shown on the left side ofFIG. 16 are pseudo images each of which is generated on the basis of themedical image G0, and are input to the learning model 4 as data forlearning. The pseudo images P1 and P2 are images in which the lightnessof the medical image G0 is brightened and darkened, respectively. Thepseudo image P3 is an image in which the contrast of the medical imageG0 is weakened. The pseudo images P4 and P5 each are an image in which apart of the medical image G0 is darkened.

Further, on the right side of FIG. 16, for each of the input pseudoimages, the region of interest extracted by the learning model 4 duringlearning is surrounded by a thick line and shown. For the pseudo imagesP1 and P2, regions of interest are appropriately extracted. For thepseudo image P3, an error “unextractable” is output. For the pseudoimages P4 and P5, a part of the unclear region that should be originallyextracted as the region of interest is not extracted as the region ofinterest.

As shown in FIG. 16, the pseudo image may have pixel values changed forthe entire medical image G0, or may have a pixel value changed for apart of the medical image G0. Further, as shown in the pseudo images P4and P5, it is preferable to use, as data for learning, plural pseudoimages generated in a case where a pixel value in each of differentregions for one medical image G0 is changed. According to such anaspect, the number of original images can be reduced, so that learningcan be performed efficiently.

Further, as shown in the pseudo images P1 to P5, the pseudo image ispreferably an image generated in a case where the pixel value is changedso that the image quality is degraded, in at least a part of the medicalimage G0. Specifically, the phrase “so that the image quality isdegraded” means processing of making it difficult to detect thestructure included in the medical image G0, and examples thereof includeprocessing of weakening the contrast. This is because it is consideredthat considering the operational phase of the learning model, in a casewhere the input medical image is an image of the fundus of the eye, forexample, image darkening caused by insufficient light or imagebrightening caused by ambient light makes the contrast weaker than thatof the clear medical image.

Further, as shown in FIG. 16, the pseudo image is preferably an imagegenerated in a case where the pixel value of the region including atleast the region of interest in the medical image G0 is changed. This isbecause it does not matter whether or not the region except for theregion of interest is unclear.

Further, the pseudo image is preferably an image generated in a casewhere the pixel value of at least a part of the medical image G0 ischanged while the existence or nonexistence of the structure included inthe medical image G0 is maintained. This is because the learning model 4gives priority to accurately extract the region of interest even from anunclear medical image, and gives low priority to correspond to thechange in the existence or nonexistence of the structure.

Further, the pseudo image may be an image generated on the basis of themedical image G0 by using an image generation model such as generativeadversarial networks (GAN) and a variational autoencoder (VAE).

The CPU 21 causes the learning model 4, to perform learning in a casewhere the pseudo images P1 to P5 generated as described above are inputto the learning model 4, as the data for learning. Further, as shown inthe pseudo images P3 to P5 of FIG. 16, in a case where the learningmodel 4 fails to appropriately extract the region of interest from theinput pseudo image, the CPU 21 may cause the learning model 4, toperform re-learning in response to the re-input of the pseudo images asthe data for learning. According to such an aspect, the accuracy of thelearning model 4 can be improved.

In a case of re-learning, a ground-truth label may be given to thepseudo image. Specifically, the CPU 21 may cause the learning model 4,to perform re-learning in a case where a pair of the pseudo image andinformation indicating the region of interest included in the pseudoimage is input to the learning model 4 as the data for learning. Theinformation indicating the region of interest means informationindicating, for example, the position of the region of interest in thepseudo image.

Next, an operation of the information processing apparatus 10 accordingto the present embodiment will be described with reference to FIG. 17.The CPU 21 executes the information processing program 27, so thatlearning processing shown in FIG. 17 is executed. The learningprocessing shown in FIG. 17 is executed, for example, in a case wherethe user gives an instruction on the start of the processing via theinput unit 25.

In Step S71, the CPU 21 acquires the original image obtained by imagingthe subject, from the image acquisition apparatus 2. In Step S72, theCPU 21 changes the pixel value of at least a part of the original imageacquired in Step S71 to generate a pseudo image. In Step S73, the CPU 21causes the learning model 4 to perform learning by using the pseudoimage generated in Step S72 as data for learning. In Step S74, the CPU21 determines whether or not the learning model 4 can appropriatelyextract the region of interest from the input pseudo image.

In a case where the learning model 4 fails to appropriately extract theregion of interest from the input pseudo image (that is, negativedetermination is made in Step S74), the process returns to Step S73, andthe CPU 21 causes the learning model 4, to perform re-learning inresponse to the re-input of the pseudo image to the learning model 4.That is, the re-learning using the same pseudo image is repeated untilthe learning model 4 can appropriately extract the region of interestfrom the pseudo image. The learning processing ends at the timing whenthe learning model 4 appropriately extracts the region of interest fromthe input pseudo image (that is, the timing when affirmativedetermination is made in Step S74).

As described above, the information processing apparatus 10 according tothe fourth embodiment comprises at least one processor, and theprocessor causes the learning model that is used to extract a region ofinterest from an input image, to perform learning in response to aninput of the pseudo image generated in a case where the pixel value ofat least a part of the original image obtained by imaging the subject ischanged, as the data for learning. Therefore, even in a case of an imageunsuitable for extraction of a region of interest, a region of interestcan be appropriately extracted and the image can be utilized fordiagnosis.

In the fourth embodiment, an aspect in which the information processingapparatus 10 causes the learning model 4 that is used by the extractionunit 12 to perform learning by unsupervised learning has been described,but the present disclosure is not limited thereto. The informationprocessing apparatus 10 may cause the learning model 4 to performlearning by supervised learning and semi-supervised learning.Specifically, the CPU 21 may cause the learning model 4, to performlearning in response to the input of a pair of the pseudo image andinformation indicating a region of interest included in the pseudo imageas data for learning. Further, in this case, in a case where thelearning model 4 fails to extract the region of interest from the inputpseudo image, the CPU 21 causes the learning model 4, to performre-learning in response to the re-input of the pseudo image as the datafor learning.

In the above-described embodiments, description has been made by usingthe medical image, but the technology of the present disclosure is notapplied only to the medical image. The technology of the presentdisclosure may be applied to an image that is acquired by using, as asubject, a device, a building, a pipe, a welded portion, and the like ina non-destructive inspection such as a radiographic inspection and anultrasonic inspection.

Further, in each of the above-described embodiments, an aspect in whichthe information processing system 1 includes the information processingapparatus 10 and the image acquisition apparatus 2 has been described,but the present disclosure is not limited thereto. For example, theinformation processing system 1 may include one apparatus having boththe function of the information processing apparatus 10 and the functionof the image acquisition apparatus 2. Alternatively, for example, theinformation processing system 1 may include plural the image acquisitionapparatuses 2, and the information processing apparatus 10 may acquire amedical image from each of the plurality of image acquisitionapparatuses 2. Alternatively, for example, the information processingapparatus 10 may consist of plural devices that are different from eachother for each function, such as the acquisition unit 11, the extractionunit 12, the determination unit 13, the detection unit 14, thepresentation unit 15, and the combination unit 16.

Further, in the above-described embodiment, for example, as a hardwarestructure of a processing unit that executes various processing such asprocessing performed by the acquisition unit 11, the extraction unit 12,the determination unit 13, the detection unit 14, the presentation unit15, and the combination unit 16, the following various processors may beused. The various processors include, for example, a programmable logicdevice (PLD), such as a field programmable gate array (FPGA), which is aprocessor having a changeable circuit configuration after manufacture,and a dedicated electrical circuit, such as an application specificintegrated circuit (ASIC), which is a processor having a dedicatedcircuit configuration designed to perform specific processing, inaddition to the CPU which is a general-purpose processor that executessoftware (program) to function as various processing units as describedabove.

One processing unit may be constituted of one of the various processorsor may be constituted of a combination of two or more processors of thesame type or different types (for example, a combination of plural FPGAsand a combination of a CPU and an FPGA). Further, the plurality ofprocessing units may constitute one processor.

A first example of the configuration in which the plurality ofprocessing units are constituted of one processor is an aspect in whichone or more CPUs and software are combined to constitute one processorand the processor functions as plural processing units. A representativeexample of the aspect is a computer such as a client and server. Asecond example of the configuration is an aspect in which a processorthat implements all of the functions of a system including the pluralityof processing units with one integrated circuit (IC) chip is used. Arepresentative example of the aspect is a system-on-chip (SoC). Asdescribed above, as the hardware structure of various processing units,one or more of the various processors are used.

Furthermore, as the hardware structure of the various processors, morespecifically, an electrical circuit (circuitry) in which circuitelements, such as semiconductor elements, are combined may be used.

In the present embodiment, an aspect in which the information processingprogram 27 is stored (installed) in the storage unit 22 in advance hasbeen described, but the present disclosure is not limited thereto. Theinformation processing program 27 may be recorded on a recording medium,such as a compact disc read only memory (CD-ROM), a digital versatiledisc read only memory (DVD-ROM), or a universal serial bus (USB) memory,and then provided. Further, the information processing program 27 may bedownloaded from an external apparatus via the network. Furthermore, thetechnique of the present disclosure extends to a storage medium on whichthe information processing program is non-temporarily stored, inaddition to the information processing program.

In the technology of the present disclosure, the above-describedexemplary embodiments can be appropriately combined with each other. Thecontents described and illustrated above are detailed descriptions forthe part related to the technology of the present disclosure, and aremerely an example of the technology of the present disclosure. Forexample, the description regarding the above-described configuration,function, operation, and effect is the description regarding an exampleof the configuration, function, operation, and effect of the partaccording to the technology of the present disclosure. Accordingly, itgoes without saying that an unnecessary part may be deleted, a newelement may be added, or replacement may be made with respect to thecontents described and illustrated above, within a scope not departingfrom the gist of the technology of the present disclosure.

What is claimed is:
 1. An information processing apparatus comprising atleast one processor, wherein the processor is configured to: acquire afirst image obtained by imaging a subject; extract a first region ofinterest from the first image; and determine whether or not anunsuitable region that is unsuitable for extraction exists, for theextracted first region of interest.
 2. The information processingapparatus according to claim 1, wherein the processor is configured tomake the determination on the basis of a degree of similarity between ashape of the extracted first region of interest and a predeterminedreference shape for the first region of interest.
 3. The informationprocessing apparatus according to claim 1, wherein the processor isconfigured to make the determination, by using a learned model that isused to determine whether or not the unsuitable region exists in aregion of interest which is extracted from an image obtained by imaginga subject, in response to an input of the image.
 4. The informationprocessing apparatus according to claim 3, wherein the learned model isa learning model of which learning has been performed by using, as datafor learning, a pair of an image obtained by imaging a subject andinformation indicating whether or not the unsuitable region exists in aregion of interest which is extracted from the image.
 5. The informationprocessing apparatus according to claim 1, wherein the processor isconfigured to specify and present the unsuitable region in the firstimage in a case where the processor determines that the unsuitableregion exists in the first region of interest.
 6. The informationprocessing apparatus according to claim 1, wherein the processor isconfigured to present a ratio of the unsuitable region to the firstregion of interest in a case where the processor determines that theunsuitable region exists in the first region of interest.
 7. Theinformation processing apparatus according to claim 1, wherein theprocessor is configured to re-extract the first region of interest withreduced extraction accuracy in a case where the processor determinesthat the unsuitable region exists in the first region of interest. 8.The information processing apparatus according to claim 1, wherein theprocessor is configured to: detect a structure included in the firstregion of interest; and detect a structure included in the unsuitableregion with reduced detection accuracy in a case where the processordetermines that the unsuitable region exists in the first region ofinterest.
 9. The information processing apparatus according to claim 1,wherein the processor is configured to request a second image includinga region corresponding to at least a part of the unsuitable region in acase where the processor determines that the unsuitable region exists inthe first region of interest.
 10. The information processing apparatusaccording to claim 9, wherein the processor is configured to: acquirethe second image; and combine the first image and the second image togenerate a third image.
 11. The information processing apparatusaccording to claim 10, wherein the processor is configured to combineany one image of the first image or the second image with a part of theother image so that the unsuitable region in the one image iscomplemented by a corresponding region in the other image, to generatethe third image.
 12. The information processing apparatus according toclaim 10, wherein the processor is configured to select and combine oneimage having better image quality for each plurality of sections in thefirst image and the second image, to generate the third image.
 13. Theinformation processing apparatus according to claim 10, wherein theprocessor is configured to: extract a third region of interest from thethird image; determine whether or not the unsuitable region exists, forthe extracted third region of interest; and repeat acquisition of a newimage including a region corresponding to at least a part of theunsuitable region and re-combination of the new image and the thirdimage until the processor determines that the unsuitable region does notexist in the third region of interest.
 14. The information processingapparatus according to claim 10, wherein the processor is configured to:extract a third region of interest from the third image; and detect astructure included in the third region of interest.
 15. The informationprocessing apparatus according to claim 9, wherein the processor isconfigured to: acquire the second image; extract a second region ofinterest from the second image; detect a structure included in each ofthe first region of interest and the second region of interest; andcombine detection results of the structures that are detectedrespectively from the first region of interest and the second region ofinterest.
 16. The information processing apparatus according to claim15, wherein the processor is configured to: determine whether or not acommon unsuitable region that is unsuitable for extraction in commonexists, for the extracted first region of interest and second region ofinterest; and repeat acquisition of a new image including a regioncorresponding to at least a part of the common unsuitable region,extraction of a region of interest from the new image, detection of astructure included in the region of interest, and re-combination of adetection result of the structure, until the processor determines thatthe common unsuitable region does not exist.
 17. The informationprocessing apparatus according to claim 1, wherein the first region ofinterest is a region including at least one of the subject, a part of atissue included in the subject, or an abnormal part included in thesubject or the tissue.
 18. The information processing apparatusaccording to claim 1, wherein the first image is an image obtained by atleast one of a radiography apparatus, a magnetic resonance imagingapparatus, an ultrasonic apparatus, a fundus photography apparatus, oran endoscope.
 19. An information processing method executed by acomputer, the method comprising: acquiring a first image obtained byimaging a subject; extracting a first region of interest from the firstimage; and determining whether or not an unsuitable region that isunsuitable for extraction exists, for the extracted first region ofinterest.
 20. A non-transitory computer-readable storage medium storingan information processing program causing a computer to execute aprocess comprising: acquiring a first image obtained by imaging asubject; extracting a first region of interest from the first image; anddetermining whether or not an unsuitable region that is unsuitable forextraction exists, for the extracted first region of interest.